A/B testing (also known as split testing or bucket testing) is a method of comparing two versions of a system against each other to determine which system performs better with respect to one or more selected metrics. A/B testing presents two or more variants of a system to a selection of users at random and statistical analysis is used to determine which variation performs better for a given system goal. In some instances, A/B testing is performed between a current production version of a system and a potential implantation of a system.
In order to determine system performance, A/B testing requires enrolling users in experiments and presenting different elements to those users. A user may be provided either a current version (control) or a variation of the current version (experimental) and the user engagement with each version is measured. The measurement can occur at one or more measurement points defined by the system. In some instances, the measurement points correspond to user interactions with the system. A predetermined number of users is required for comparison and testing purposes between the control and experimental versions.